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I . SUMMARY 


Research  performed  under  Contract  N00014-75-C-0779  was  completed  March  3, 
1978.  This  research  has  explored  probabilistic  and  statistical  structures  of 
the  Linear-Quadratic-Gaussian  class  of  control  and  estimating  systems  for  pur- 
poses of  development  and  enhancement  of  design  procedures.  En  route  a complete 
theory  of  "performance  analysis"  has  been  developed  for  this  problem  class. 

Performance  analysis  allows  the  system  designer  to  see  the  quality  of  a 
given  design  in  a complete  statistical  sense.  As  such,  it  becomes  part  of  a 
computer-aided  design  procedure  but,  unfortunately,  does  not  explicitly  tell 
the  designer  how  to  modify  the  design  to  achieve  better  performance.  Thus,  the 
research  effort  has  recently  focused  on  the  "selection"  aspect  of  design. 

A complete  theory  of  control  selection  has  not  been  established  in  this 
research  effort;  however,  significant  progress  has  been  made  toward  a complete 
theory.  In  particular,  formulas  for  statistics  of  LQG  design  performance  mea- 
sures for  both  continuous  and  discrete  time  systems  have  been  derived.  Also, 
a control  selection  technique  based  upon  this  formulation  has  been  developed. 
This  control  selection  technique,  when  coupled  with  the  performance  analysis 
technique,  becomes  an  effective  design  procedure  for  stochastic  control  systems. 
An  inherent  property  of  the  resulting  designs  is  that  they  all  contain  estima- 
tors in  their  feedback  structures  and  exhibit  a separation  property. 

Two  interim  technical  reports  were  distributed  on  the  performance  analysis 
and  control  selection  aspects.  These  were: 

[1]  S.  R.  Liberty  and  R.  C.  Hartwig,  "On  the  Essential  Quadratic  Nature 
of  LQG  Control -Performance  Measure  Cumulants",  Interim  Technical 
Report-Contract  N00014-75-C-0779,  Texas  Tech  University,  August  1975, 
and 

[2]  R.  C.  Hartwig,  "Design  of  Stochastic  Linear  Control  Systems  According 
to  Cumulant  Based  Performance  Criteria",  Interim  Technical  Report- 
Contract  N00014-75-C-0779,  Texas  Tech  University,  May  1976. 

Report  [1]  also  appeared  under  the  same  title  in 


T 


1 


I 

\ 


[3]  Information  and  Control,  Vol . 32,  No.  3,  November  1976. 


Work  related  to  [1]  and  portions  of  [2]  have  appeared,  or  will  appear,  in 

[4]  0.  M.  Brown,  R.  C.  Hartwig  and  S.  R.  Liberty,  "On  the  Oesign  and  Per- 
formance Analysis  of  Linear  Stochastic  Regulators",  Proc.  of  the  7th 
Asilomar  Conference  on  Circuits  and  Systems,  pp.  563-566,  November 
1973, 

[5]  R.  C.  Hartwig  and  S.  R.  Liberty,  "A  New  Class  of  Statistical  Perfor- 
mance Criteria  for  Stochastic  Linear  Control  Systems",  Proc.  of  the 
IEEE  Conference  on  Decision  and  Control,  pp.  1136-1141,  November 
1976, 


[6]  S.  R.  Liberty,  "Performance  Analysis  of  Stochastic  Linear  Control 

Systems:  A New  Viewpoint",  Proc.  of  the  International  Forum  on  Alter- 
natives for  Multivariable  Control,  pp.  79-86,  October  1977, 


and  A Co^F^ts^ 

[7]  S.  R.  Liberty  and  R.  C.  Hartwig, ^besign-Performance-Measure  Statis- 
tics for  Stochastic  Linear  Control  Systems^;. to  appear  in  IEEE  Trans- 
actions on  Automatic  Control,  Vol.  AC-23,  No/  6,  December  1978, 


which  is  included  here  as  Appendix  A. 


Also,  the  following  theses  on  these  topics  have  been  completed: 


[8]  R.  C.  Hartwig,  "Cumulants  of  an  IQF  Differential  Equations",  Master 
of  Science  Thesis,  Texas  Tech  University,  December  1973, 

[9]  J.  M.  Brown,  "A  Computer-Aided  Design-Analysis  Technique  for  Stochas- 
tic Linear  Regulator  Problems",  Master  of  Science  Thesis,  Texas  Tech 
University,  August  1973, 


[10]  R.  C.  Hartwig,  "Design  of  Stochastic  Linear  Control  Systems  According 
to  Cumulant  Based  Performance  Criteria",  Ph.D.  Dissertation,  Texas 
Tech  University,  May  1976. 

Another  thesis  was  completed  on  additional  topics  involving  Riccati  equations 
arising  In  cumulant  based  design  procedures  for  both  control  and  estimating  sys- 
tems. This  work, 

[11]  M.  L.  Yao,  ^Initial  Studies  of  Riccati  Equations  Arising  in  Stochas- 
tic Linear  System  Theory^,  Master  of  Science  Thesis,  Texas  Tech  Uni- 
versity, December  1977,  *r 

is  included  here  as  Appendix  B.  A second  Ph.D.  dissertation  on  optimal  cumu- 


lant control  selection  in  discrete  time  systems  is  in  the  final  stages  of  prep- 
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aration  and  will  appear  as  an  interim  report  on  Contract  N00014-78-C-0443. 
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DESIGN-PERFORMANCE-MEASURE  STATISTICS  FOR 


STOCHASTIC  LINEAR  CONTROL  SYSTEMS* 


STANLEY  R.  LIBERTY,  Member,  IEEE,  and 
RONALD  C.  HARTWIG,  Member,  IEEE 


ABSTRACT  - Formulas  for  statistics  of  the  standard  integral 
quadratic  performance  measure  used  in  stochastic 
linear  control  system  design  are  derived.  The 
formulas  are  expressed  in  terms  of  dynamical 
variables  under  the  usual  assumptions  on  noise, 
plant,  and  admissibility  of  control.  All  of 
these  dynamical  variables  are  expressed  as 
linear  transformations  of  plant  state  estimates. 
The  practicality  of  this  work,  which  is  directed 
toward  the  establishment  of  new,  statistically 
based  design  procedures  for  stochastic  linear 
systems,  is  demonstrated  by  example. 
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I.  INTRODUCTION 


Once  a state  model  for  a stochastic  linear  system  is  obtained, 
the  analytical  aspects  of  designing  a feedback  controller  can  be 
conceptually  dichotomized.  We  refer  to  the  first  part  as  "control 
selection"  and  to  the  second  as  "performance  analysis." 

In  the  control  selection  part  the  designer  chooses  a performance 
measure  that  reflects  a priori  design  specifications.  Statistical 
indices  associated  with  this  performance  measure  are  then  specified 
and,  finally,  a controller  is  attained  via  optimization  of  these  in- 
dices. 

Performance  analysis  involves  determination  of  how  a specific 
controller  behaves.  This  is  accomplished  by  computing  statistical  or 
probabilistic  descriptions  of  performance  measures  that,  in  general, 
are  different  than  the  performance  measure  chosen  in  the  control 
selection  part  of  the  design  procedure.  For  example,  these  measures 
may  be  selected  to  give  the  designer  specific  insight  Into  how  well 
the  controller  is  regulating  or  how  much  control  effort  is  being  ex- 
pended. 

For  certain  classes  of  stochastic  linear  control  systems  the  per- 
formance analysis  problem  has  been  solved;  see  [7]  and  [14].  However, 
the  control  selection  aspect  of  the  design  problem  has  only  been  solved 
in  special  cases.  For  the  class  of  systems  treated  in  this  paper, 
namely  the  Linear-Quadratic-Gausslan  (LQG)  class,  a feedback  control- 
ler that  minimizes  the  mean  of  an  Integral  quadratic  performance  measure 


has  been  found  [16],  An  open-loop  controller  that  minimizes  the  variance 
of  an  integral  quadratic  performance  measure  subject  to  a constraint  on 
its  mean  has  also  been  discovered  [9],  Mo],  [121,  [13],  but  feedback 
solutions  in  the  context  of  performance  measure  statistics  beyond  the 
mean  have  not  surfaced.  The  reason  for  this  is  a lack  of  tractable 
higher  order  statistical  descriptions  of  the  performance  measure. 

In  this  paper  we  present  the  formulation  of  a complete  set  of 
statistics  of  the  integral  quadratic  performance  measure  normally  em- 
ployed in  stochastic  linear  control  system  design.  These  results  are 
an  outgrowth  of  work  initiated  a little  over  ten  years  ago  by  Sain  [11]. 
Our  formulation  is  expressed  in  terms  of  dynamical  variables  related  to 
an  estimate  of  the  plant  state  and  should  lead  the  way  to  new  classes  of 
feedback  control  structures  in  the  stochastic  linear  context. 

We  have  attempted  to  keep  our  notation  as  compatible  as  possible 
with  that  in  the  tutorial  paper  by  Tse  [15]  and  strongly  recommend  that 
this  reference,  along  with  [8],  be  primarily  used  by  the  reader.  Our 
presentation  is  formal  so  no  special  mathematical  skills  should  be  re- 
quired for  a fundamental  understanding  of  the  material. 

In  Section  II  we  describe  the  system,  the  performance  measure  and 
the  control  objective.  Section  III  contains  the  development  of  a complete 
statistical  description  of  this  performance.  An  example  of  control  system 
design  utilizing  the  new  formulation  is  presented  in  Section  IV. 


* I 


« 


\ 


r 


II.  THE  SYSTEM  DESCRIPTION  AND  PERFORMANCE 

Let  Rp  denote  the  p-fold  Cartesian  product  of  the  real  line,  and  let 
I denote  the  real  line  interval  [tQ,  t^].  We  wish  to  control  the  noisy 
linear  system  described  on  I by 

» A(t)x(t)  + B(t)u(t)  + 5(t)  (1) 

and 

z(t)  = C(t)x(t)  + 0 ( t ) , (2) 


where  the  state  x(t)  eRn,  the  control  action  u(t)  £Rm,  and  the  observation 
z(t)  eRr.  The  initial  condition  for  (1),  x(tQ),  is  assumed  to  be  Gaussian 
with  mean 


xo  = 

E{x(to)} 

(3) 

and  covariance 

E0  3 E{[x(to)_xo 

][xT(t0)-xJ]} 

(4) 

where  denotes  matrix  transposition.  The  state  process 

the  observation  noise,  0,  are  zero-mean  Gaussian-white  with 

noise,  £» 

and 

EU(t)eV)} 

= 0, 

t ,xel, 

(5) 

E{[x(t0)-x0]^T(t)} 

- 0, 

tel , 

(6) 

E{[x(t0)-x0]eT(t)} 

- 0, 

tel , 

(7) 

E{C(t)^T(x)> 

= E(t)6(t-T ) , 

t,iel , 

(3) 

and 

E{0 ( t )eT (t) } 

= 0(t)6(t-T), 

t , T€ I , 

(9) 

•! 

* 
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where  E(t)  and  0(t)  are  positive  semi-definite  and  positive  definite,  respectively, 
on  I. 

We  require  that  the  control  action,  u,  be  a causal  function  of  the  ob- 
servation. That  is, 

u(t)  = <p(t,  z( T):re[t0,t))  tel,  (10) 

where  p satisfies  certain  technical  assumptions  stated  in  [15];  also 
see  [16].  All  matrix  functions  on  I and  the  mapping  p are  assumed  to 

be  smooth  enough  to  guarantee  mean-square  continuity  of  the  state  process  on  I. 

For  design  purposes  we  define  a "design-performance-measure" 

J = xT(tf)Sx(tf ) + / f [xT(t)Q(t)x(t)  + uT(t)R(t)u(t)]dt,  (11) 

fco 

where  the  terminal  penalty  weighting,  S,  is  symmetric  and  positive  semi- 
definite  as  is  the  weighting  Q(t)  on  I.  The  weighting  R(t)  is  symmetric 
and  positive  definite  on  I,  and  both  Q(t)  and  R(t)  are  continuous  on  I. 

These  weighting  matrices  are  given  values  by  the  desianer  that  reflect 
a priori  design  specifications  involving  the  relative  importance  of  state 
regulation  and  control  effort.  The  design  objective  is  to  choose  u 
in  (10)  so  system  performance  is  "good"  in  some  sense. 

The  functional,  J,  assigns  a non-negative  real  number  to  each  sample  run 
of  the  control  system  with  small  values  implying  good  performance.  However, 
the  question  of  quality  of  performance  is  multiply  clouded.  First,  J is  random 
so  it  is  only  meaningful  to  refer  to  J in  a statistical  or  probabilistic  sense. 
Second,  since  J is  the  sum  of  terms  representing  measures  of  state  regulation 


4- 


and  control  effort,  the  individual  quality  of  these  measures  is  not  apparent  in  a 
broad  statistical  description  of  J.  We  will  not  address  these  subtleties  here, 
but  will  concentrate  on  obtaining  a statistical  description  of  J that  might  be 
used  as  a basis  for  selection  of  control  action.  In  Section  IV,  the  concepts  of 
control  selection  and  performance  analysis  are  demonstrated  by  example. 
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III.  A COMPLETE  STATISTICAL  DESCRIPTION  OF  J 


Let  be  the  sigma-algebra  induced  by  the  observation  (z(t) :Te[tQ,a) }. 
When  a = t^  we  will  write  F without  a subscript.  We  will  now  generate  con- 
ditional statistics  of  J . Expand  the  process  modeled  by  (1)  in  an  orthonormal 
series , 


x(t)  ~ Z x 4>  (t), 
i=l  1 1 


tel  , 


02) 


where  the  x^s  are  scalar  random  variables  given  by 


xt  = xT(tf)  S<Di(tf)  + /f  xT(t)Q(t)g>1(t)dt , vi, 

lo 


03) 


and  the  orthonormality  constraint  on  the  nonrandom,  vector-valued  <(i.'s  is 
given  by 


♦i(tf)S*j(tf)  + /tf4>i(t)Q(t)4>j (t)dt  - 6±j  , Vi,  j, 


04) 


In  addition,  we  require  that  the  x^s  be  conditionally  uncorrelated,  that  is 


E{[xi-mi][xj-m.]|F}  = X^,  Vi,  j,  (15) 

where  mi  is  the  conditional  mean  of  xt  given  by 
mt  = E(xT(tf)jF}  S4>1(tf) 

T 

+ r E(x  (t)  | F}Q( t )<J> . (t)dt,  Vi,  j.  (16) 

A.  ^ 


-6- 


A necessary  and  sufficient  condition  for  (15)  given  (13)  and  (14)  is 


i 


i 

t 


i 

i 


that  $ and  x,  satisfy 

Xi4)i(t)  = r r(t.T)Q(T)0i(T)dr  + r(t,tf)  S«Di(tf),  tel,  Vi,  (17) 
lo 

where  r is  the  smoothed-estimate,  error-covariance  kernel  of  the  state 
process.  That  is,  let  the  smoothed  estimate  of  x(t)  be  denoted  by 


x(t|tf)  = E{x(t) | F} , tel . 


(18) 


Then  r is  given  by 

F(t,r)  = E{[x(t)  - x(t|tf)]  [xt(t)-xT (r|tf )]jF}  t.rel.  (19) 

As  a consequence  of  the  1 inear-Gaussian  assumptions  of  Section  II  and  the 
technical  assumptions  on  </> , 

r(t,x)  = E{r(t,r)}  • (20) 

That  is,  r is  nonrandom  implying  that  each  x^  is  nonrandom. 

Under  the  assumptions  we  have  made,  J is  finite  with  probability  one; 
see  Doob  [1].  It  follows  that  the  series  in  (12)  converges  in  the  square 
integrable  sense.  Thus, 

J * ? x2  + / uT(t)R(t)u(t)dt,  (21) 

i-1  i tQ 

with  probability  one;  see  Kolmogorov  and  Fomin  [5J. 
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Since  x is  conditionally  Gaussian  each  xi  is  conditionally  Gaussian. 

But,  we  have  forced  the  xi's  to  be  conditionally  uncor^elated,  thus  they  are 
conditionally  independent  as  are  their  squares.  The  conditional  characteristic 
function  of  each  xf  term  in  (21)  is  of  the  noncentral  chi-square  type  given  by, 


Cx2jF(j«/)  = (l-2j«/Ai)"l5exp[jwm£i(l-2jw^)‘'L]. 


(22) 


The  conditional  characteristic  function  of  J follows  as. 


C.,r(jw)  = [ff  (l-2wX  rt  • exp[jw  ff  uT(t)R(t)u(t)dt 
U|  r ,_i  i r 

tO 


i-1 


+ ? jum^(l-2jwx  r1]. 

i-l  1 


(23) 


In  our  previous  work  [6  ],  [7  ] we  have  observed  that  in  the  Linear-Quadratic- 
Gaussian  class  of  systems  the  second  characteristic  function  generates  trac- 
table statistical  forms.  The  second  conditional  characteristic  function, 

Tj | p,  is  defined  as  the  natural  logarithm  of  Cj|p,  that  is, 

Tj  j p(jai)  » ln[Cj|p(J«)].  (24) 


The  formal  MacLaurin  series  representation  of  Tj|p  is 


oo 

Tj|p(jo»)  = l 
1 1-1 


:i|F 


LR1 

i! 


(25) 


where  the  coefficients  <i|p  are  called  conditional  cumulants.  Utilizing  (23), 
it  can  be  easily  shown  that  the  first  conditional  cumulant  of  J is  given  by 
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(26) 


<1 1 F = 1f1  Xt  + if1  m2  + ;f  uT(t)R(t)u(t)dt, 

1 lo 


while  the  remaining  conditional  cumulants  are  of  the  form 


'k|F 


= (k-l)!2 


k-1 


T Xk.  + 

i-l  i 


k ! 2 


k-1 


I m2Xk"| 

i«l  i i 


k > 1 . 


(27) 


Although  the  conditional  cumulants  as  given  by  (26)  and  (27)  are  complete  in 

the  sense  that  any  statistic  of  J can  be  derived  from  them,  they  are  not  in  an 
attractive  form  for  the  control  system  designer  since  they  are  not  expressed  in 
terms  of  system  variables.  To  accomplish  this  we  must  attack  the  series  ex- 
pressions in  (26)  and  (27).  The  first  step  is  to  define  "iterated  kernels" 


and 


r(k)  (t ,t ) & r(t,  tf)  sr(k_1)(tf,T) 


r(1)  ( t ,t ) ^ r(t,T) 


(k-1), 


+ £ T(t,a)  Q(a)T^k'1 ^(a.T)  do,  k > 1 


It  can  be  inductively  shown  using  (14)  and  (17)  that 


r^(t,x)  * X^th^T). 


k . / \ J i 


It  follows  that  the  expression,  Xk  , can  be  written  as 

Xk  = Tr[ST^k)(tf,tf)  + ff  Q(t)T^(t,t)  dt],  k * 1 , 


(28) 


(29) 


(30) 


(31) 
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where  Tr  [•]  denotes  the  trace  of  the  enclosed  matrix.  Utilizing  (16),  (18) 
and  (30)  it  follows  that 


= xT(t:f | tf ) sr(k'1}  (tf,tf)  Sx(tf|tf) 


+ iT(tf|tf)s  ;f  r(k_1)(tf,t)Q(t)i(t|tf)  dt 


+ ff  x(t|tf)Q(t)  r(k_1)(t,  tf)dt  Sx(tf|tf) 
to 


+ /f  /f  xT(t/  tf  )Q(t)r(k-1  )(t,T)Q(T)x(x)t^)dTdt,  k>l.  (32) 


tA  t„ 
0 0 


For  the  case,  k=l , it  is  easily  seen  that 


il1  mZ±  * xT(tf  |tf  )Sx(tf  |tf) 


+ ff  xT(t|tf )Q(t)x(t|tf )dt. 

4o 


Consider  the  last  term  in  (32)  and  note  that  it  contains  a symmetric  (in 
argument)  integrand.  Therefore  it  may  be  rewritten  as 


-10- 


V'T , 


/r  /rx 1 (t|tf)Q(t)r(k'1)(t,T)Q(T)x(T|tf)dTdt 


0 0 


= 2 /fxT(t|tf )Q(t)  / r^k-1 ^(t,T)Q(x)x(T|tf)dTdt 

t T t T 

o o 


(34) 


Define  the  new  n-vector  valued  variable 


n^U)  - / r^k  1 ^ ( t ,t)Q(t)x(t  | tf  )dt 


(35) 


and  note  that 


nk-l(V  = °- 


(36) 


The  conditional  cumulants  can  now  be  written  as 


‘1|F 


*T(tf|tf)  Sx(tf | tf ) + /f[xT(t|tf)q(t)x(t|tf) 

to 


+ u (t)R(t)u(t)]dt 


(37) 


+ Tr[ST(tf ,tf ) + ff  Q(t)r(t,t)dt], 
to 


and 


:k|F  =k!2k-1[XT(tf|tf)Sr(k-1)(tf,tf)Sx(tf|tf) 

+ 2xT(tf|tf)Snk.1(tf)  + 2 /f  xT(t|tf)Q(t)nk.-,(t)dt] 

lo 

+ (k-l)!2k'1Tr[Sr(k)(tf,tf)  + / Q(t)r(k) (t.t)dt] , k > 1 


(38) 
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Equations  (37)  and  (38)  provide  us  with  the  conditional  cumulants  of  the 
design-performance  measure,  J,  expressed  in  terms  of  the  smoothed  esti- 
mate of  the  x-process  and  the  corresponding  error  covariance  kernel. 

Any  statistic  of  J can  be  expressed  in  terms  of  the  conditional  cumulants 
via  their  relationship  to  conditional  moments.  This  relationship  is  of 
the  same  form  as  (42)  below.  For  example,  denoting  a cumulant  of  J by 
< we  have 


E { J } = <1 

= E(,c1 1 F>- 

(39) 

Var(J)  = <2 

= + Var{<.|  |F>, 

(40) 

and  in  general 

= E{k^ , p}+  (statistics  of  lower  order  conditional  cumulants).  (41) 


The  relationship  between  noncentral  moments  and  cumulants  is  well  tabulated 
[4].  Denoting  a noncentral  moment  of  J by  u|<*  this  relationship  is  given  by 


k k 

Wk+1  = jf0{J )uk-j<j+l  . 


(42) 


Those  familiar  with  the  traditional  minimum  mean  LQG  problem  may 
be  a little  suspicious  of  equation  (37)  since  it  is  well  known  that 
E{ <1 | p>  is  normally  expressed  in  terms  of  the  filtered  estimate  [3],  x(t|t)» 
and  its  corresponding  error  covariance  with  precisely  the  same  structure  as 
(37)  under  expectation;  see  [15].  To  demonstrate  the  equivalence  of  these 
two  formulations  note  that  in  view  of  (10)  the  smoothed  estimate  can  be  ex- 
pressed as 


-12- 


x(t|t^)  = x ( 1 1 1 ) + f K(t, t)  C^(t)0"^(t)  C(t)  v(t | t )dr , tel,  (43) 


see  [ 8],  where 


x ( t | t ) = E{x(t)|Ft}, 


(44) 


K (t,i)  = E{[x(t)  - x(t|t)][xT(x)  - xT(t|t)]}  , 


(45) 


and  the  "innovation"  [2],  v,  is  given  by 


v(t|t)  = C(t)[x(t)  - x(t|t)]  + 9(t). 


(46) 


The  smoothed  error  covariance  can  also  be  expressed  as 


r(t,t)  =K(t ,t)-/f  K(t,a)CT(o)e"1.(o)  C(a)K(a,t)do 

tVT 


where  tvx  means  max  [t,t]. 


(47) 


Substitution  of  (43)  and  (47)  into  (37)  and  aoplication  of  expectation 
immediately  yields 


1 = E£<] | F>  = E{xT(tf|tf)Sx(tf|tf)  + /T[x'(t|t)Q(t)x(t|t) 


frd/ 


+ u'(t)R(t)u(t)]dt)  + Tr[SZ(tf)  + /T  Q(t)E(t)dt], 

T t 


(48) 


where 


E(t)  * K (t ,t) 


(49) 
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and  we  have  utilized  the  fact  that  v is  white  noise  with  covariance 
E{v(t|t)vT(x|x)}  = e(t)6(t-x).  (50) 

In  deriving  (48)  from  (37)  only  one  subtlety  arises  that  might  be 
troublesome  to  the  reader.  In  particular,  two  terms  of  the  form 


E{  /f  xT(t| t)Q(t)  /f  K(t,x)CT(x)e'1(x)C(x)v(x|x)dt> 

to  t 

= Tr[  f*  Q(t)  /fK(t,x)CT(x)©"1 (x)C(x)E{v(x|x)xT(t|t)}dxdt]  (51) 

*0  1 

arise. 

It  is  well  known  [14]  that  under  some  technical  assumptions  on  the  causal 
mapping  <p  in  (10) 


„ t t 

x ( 1 1 1 ) = / G(t,x)v(x|x)dx  + f <t(t,x)B(x)u(x)dx  + <t(t,t  )x  (52) 

t ' t 0 

0 0 


for  some  kernel  G where  <t>  is  the  transition  matrix  associated  with 
A in  (1).  Since  the  control  u is  assumed  to  be  a causal  function  of 
the  observation  z,  which  in  turn  can  be  expressed  as  a causal  func- 
tion of  the  innovation,  v,  it  follows  in  view  of  (50)  that  the  inner- 
most integrand  in  (51)  is  zero  almost  everywhere.  Consequently,  the 
terms  in  question  vanish  under  expectation. 
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IV.  A DESIGN  EXAMPLE 


How  might  these  formulations  be  utilized?  there  are  many  possible 
answers  to  this  question.  For  example,  let  us  assume  that  the  design 
objective  is  to  select  a controller  that  will  keep  the  variance  of  J 
small  without  the  mean  of  J becoming  too  large.  Such  an  objective 
suggests  selection  of  a weighted  sum  of  the  indices,  mean  and  variance, 
for  optimization.  Such  a criterion  is: 

minimize  [E{ic-j  | F } + ct(Ef <2 |F}  + Var(<l|F}^ 

over  all  admissible  control  laws  subject  to  the  obvious  dynamical  con- 
straints. The  nonnegative  real  parameter  a allows  a trade-off  between  mean 
and  variance  to  be  effected  in  the  design  procedure.  For  example,  when  a 

I 

is  zero  the  criterion  collapses  to  the  traditional  minimum  mean  criterion. 
When  the  designer  selects  large  a,  emphasis  is  placed  upon  making  the  vari- 
ance of  performance  small. 

The  design  procedure  consists  of  selection  of  several  controllers  via 
the  above  criterion,  with  performance  analysis  of  each  until  an  acceptable 
trade-off  between  mean  and  variance  of  performance  is  achieved. 

Unfortunately,  research  on  this  class  of  problems  has  not  evolved  to 
the  point  where  the  complete  solution  is  known.  We  can,  however,  modify 
the  criterion  to  yield  a classical  problem  formulation  that  will  lead  to 
an  interesting  class  of  feedback  control  laws. 

To  accomplish  this  we  retain  only  second  degree  terms  in  the  criterion. 
Thus,  the  term  Var {<1 | p } is  arbitrarily  dropped.  Next  we  substitute  (43) 
and  (47)  into  (35)  and  (38)  for  k = 2 and  fully  expand  the  criterion.  All 
terms  that  are  not  affected  by  control  action  or  involve  future  dynamical 
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operations  are  discarded  to  allow  simple  enforcement  of  (10).  For  sim- 
plicity, terms  containing  the  integral  expression  in  (47)  are  also  dis- 


carded. 


It  may  appear  that  this  surgery  is  rather  drastic  but  actually  much 
of  the  original  objective  has  been  retained  in  the  modified  criterion 
which  now  looks  like 

min  E{xT(tf | tf)Sx(tf | tf ) + I [x(t|t)Q(t)x(t|t)  + uT(t)R(t)u(t)]dt 


+ 4aXT(tf|tf)Sz(tf)Sx(tf(tf)  + 8aXT(tf|tf)Sn(tf) 


+ 8a  / x (t|t)Q(t)n(t)dt}» 
^o 

where 


d - 

dt  n(t) 


[ A ( t ) - l(t)CT(t)0*1(t)C(t)]n(t)  + r(t)Q(t)x(t|t),  (53) 


n(tQ)  = 0. 


Define  the  augmented  matrices 

fx(t|t) 
x(t)  $ . 


4aQ(t) 


4aQ(t) 


a S + 4aSz(t,)S  4aS 
? = t 
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m - 


B(t)  = 


ff(t)  - 


z(t)Q(t)  A(t)-z(t)C(t)0  (t)C(t) 


where 


W(t)  ■ z(t)cT(t)0_1(t) 


The  criterion  can  then  be  rewritten  as 


T _ T T 

lin  E{  x (t-)Sx(tf)  [x  (t)<7(t)x(t)  + u (t)R(t)u(t)]dt^ 


over  admissible  u subject  to 

jT(t)  = ft(t)x(t)  + !T(t)u(t)  + W(t)v(t|t), 


*(t0) 


The  solution  of  this  "accessible  state"  problem  is,  of  course,  well 


known  [15]  and  given  by 

u ( t)  = -R'1(t)BT(t)M(t)iT(t), 
where  the  2nx2n  matrix  M(t)  is  the  solution  on  1 of 


^M(t)  = -M(t)*(t)  - AT( t)M( t)  - Q ( t ) + M(t)B(t)R'1(t)FT(t)M(t), 


M(tf ) = S. 
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A Simple  Example:  Consider  the  scalar  system  described  on  [0,1]  by 


l 


; 


t 


I 


i 


i 


and 


where 


x = x + u + £ 


z = x + 0 


X = 1 
0 


and 


2=0. 

o 

Let  conditions  (5)  through  (7)  hold  with 
E{c(t)cT(x)}  = . 25<5(  t-T ) 
and 

E{e(t)eT(t)}  = .356(t-r). 


(T) 

(2') 

( 3 ' ) 

(4') 

(S') 

(9') 


Select  the  design-performance-measure 

J = / Cx2(t)  + u2(t)]dt  (11  1 ) 

o 

and  calculate,  via  (65)  and  (64),  the  feedback  controller  that  optimizes 
the  modified  criterion  for  several  values  of  «.  For  each  of  these  con- 
trollers carry  out  performance  analysis  by  computing  statistical  and 
probabilistic  descriptions  of  state  regulation  and  control  effort. 

Specifically,  obtain  such  descriptions  for  the  function-space  squared- 
norm  of  the  plant  state  trajectory  and  the  control  action  trajectory. 

That  is,  select  post  design  performance  measures 

1 

J,  3 / x2(t)dt  (67) 

A o 

and 

Ju  3 / u2(t)dt.  (68) 


The  mean  M,  and  variance,  V,  of  J,  Jx  and  are  plotted  versus  * 
in  Figures  1-3  respecti vely . Note  that  the  original  objective  of  making 
the  variance  of  J small  at  the  expense  of  larger  mean  is  achieved.  Also 
!;  note  that  both  mean  and  variance  of  plant  state  regulation  are  improved 

by  selecting  controllers  with  large  * values.  This  "good"  regulation  is 
paid  for  by  a corresponding  increase  in  the  mean  and  variance  of  control 
effort . 

Figures  4,  5,  and  6 contain  probability  densities  for  J,  Jx,  and  Ju 
respectively  that  provide  a complete  statistical  picture  of  performance. 

Some  observations  should  be  noted.  The  class  of  control  laws  generated 
by  the  criterion  is  truly  dynamical  in  that  the  feedback  law  contains  n 
dynamics  driven  by  the  filtered  estimate  of  the  plant  state.  A separation 
property  is  inherent.  The  extension  of  the  traditional  LQG  criterion  to 
include  second  order  statistical  terms  is  simply  parameterized.  By  incor- 
porating "performance  analysis"  techniques  from  [7]  into  the  design  pro- 
cedure, insight  into  controller  properties  is  enhanced. 
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V.  CONCLUSIONS 


A great  deal  of  research  remains  to  be  done.  We  have  only  scratched 
the  surface.  In  particular,  for  cumulants  of  performance  beyond  the  mean, 
it  is  not  apparent  that  the  smoothed  estimate  formulation  will  collapse 
to  a filtered  estimate  formulation  as  it  did  in  (48).  The  resulting  pres- 
ence of  noncausal  variables  is  troublesome  in  control  selection  and  moti- 
vated our  approach  to  the  example  of  Section  IV. 

Consequently,  we  have  not  presented  a complete  theory  and  feel  that 
there  may  be  a better  formulation  of  this  problem  class  than  that  pre- 
sented here.  Despite  this,  we  are  encouraged  by  these  results  since  they 
demonstrate  a richness  of  the  LQG  problem  class  that  was  not  apparent  in 
the  minimum  mean  results  of  the  previous  decade. 

The  practical  value  of  the  design  viewpoint  expressed  in  Section  IV 
should  be  apparent.  This  viewpoint  can,  of  course,  be  directly  extended 
to  include  higher  order  statistics  with  the  obvious  consequent  increase 
in  off-line  computation  and  complexity  of  feedback  structure. 
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CHAPTER  I 


INTRODUCTION 


A new  class  of  matrix  differential  equations  has 

I 

arisen  in  the  context  of  Stochastic  linear  system  design. 

These  equations  are  nonlinear  and  are  parameterized  such  that 
the  classical  Riccati  equation  is  imbedded  in  the  class.  Be- 
cause of  this, the  new  class  of  equations  is  referred  to  as 
"Riccati  type".  The  work  presented  herein  is  the  first 
s investigation  of  some  of  the  properties  of  these  equations 

and  the  systems  in  which  they  are  utilized. 

Chapter  II  contains  a summary  of  the  derivation  of 
these  new  equations.  In  Chapter  III,  the  positive  definite 
and  symmetric  properties  associated  with  this  new  class  of 
matrix  differential  equations  are  shown.  Chapter  IV  contains 

steady  state  results  for  the  scalar  case.  In  Chapter  V,  three 

« 

numerical  examples  of  linear  estimation  and  control  are  pre- 
sented in  which  the  Riccati  equations  are  utilized  as  design- 
tools.  Conclusions  and  suggestions  for  further  research  are 
contained  in  Chapter  VI. 
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CHAPTER  II 


LINEAR  STOCHASTIC  SYSTEMS  AND  RICCATI  EQUATIONS 

An  interesting  class  of  matrix  differential  equations 
arises  in  the  context  of  determining  complete  statistical 
descriptions  of  integral  quadratic  forms  in  random  processes 
generated  by  linear  dynamical  systems.  These  systems  operate 
on  Gaussian-white  noise  to  produce  vector  valued  Gauss-Markov 
processes.  The  derivation  of  these  equations  is  contained  in 
[1]  and  the  results  are  summarized  here. 

Consider  the  stochastic  linear  dynamical  system 

x(t)  ■ F(t)x(t)  + G(t)£(t),  (2-1) 

where  the  state  x(t)£Rn  (the  n-fold  cartesian  produces  of 
the  real  line)  and  the  noise  5(t)eRia.  The  process  5 (t)  is 
assumed  to  be  Gaussian  white  with  zero  mean  and  covariance 
kernel . 

E{ ? ( t) CT ( t) > - H ( t) 6 (t-t) . (2-2) 

The  initial  state  for  (2-1),  x(tQ),  is  assumed  to  be  Gaussian 
with 

E { x ( tQ ) } « xQ,  (2-3) 

and 

E<Cx(t0)-x0KxT(t0)-x0T])  - P0  ; 12-4) 

and  (2-1)  is  defined  on  a finite  interval  [tQ,tj]. 
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Attached  to  (2-1)  is  a performance  measure,  J,  defined 


J ■ xT(t^)Sx(tj)  + J xT (t) N (t) x (t) dt  , 


(2-5) 


where  S and  N(t)  are  positive  semi-definite.  In  [l]  Liberty 
and  Hartwig  show  that  the  statistics  of  J (in  particular  the 
cumulants  of  J)  can  be  expressed  explicitly  in  terms  of  a 
countable  set  of  matrix  variables  satisfying  simultaneous 
differential  equations.  The  particular  matrix  variables  of 
interest  in  this  work  are  the  H variables  of  [l]  which  evolve 
according  to 

8(8,1)  - -FT(3)H(3,1)-H(3,1)F(S)-N(8)  , Be[t0,tf], 

(2-6) 


H (8 ,k)  - -F1  (S)H(3,k)  -H(8,k)F(8) 


(2-7) 


£ H(8,  j)G(8)Q(8)G(8)H(6,k-j)  6e:[t0,tf] 

3"1  k • 2,3,4,.. 


with  boundary  conditions 

H(tf,k) 


H ( tf , k) 


(2-8) 


k-  2,3 (2-9) 


It  is  shown  in  [1]  that  the  kfc^  cumulant  of  J contains  a term 


of  the  form  XQH(t0,k)x0< 
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The  class  of  Riccati  equations  of  interest  in  this 
work  arose  in  the  work  of  Hartwig  [2]  in  the  following  way, 
Consider  a stochastic  linear  control  system 


x ( t ) 


A (t ) X (t)  + 3(t)U(t)  + G(  tK(t)  (2-10) 


where  the  control  action,  (J(t)eR^,  is  to  be  selected  such 
that  system  performance 

J - xT(tf)Sx(tf)  +J  [xT(t)Q(t)x(t)  + uT(t)R(t)u(t)  ] dt 

t (2-11) 


is  good  in  some  statistical  sense.  In  (2-11),  Q(t)  is 
positive  semi -definite  and  R(t)  is  positive  definite.  It 
is  well-known  that  if  0(t)  is  chosen  as 


d(t)  - - R A(t)B(t)K(t)x(t)  , 


(2-12) 


where  K(t)  satisfies  the  Riccati  equation 

K (t)  * - K (t)  A(t)  - AT(t)K(t)  + K(t)B(t)R*'-1  (t)BT(t)K(t) -Q(t), 

(2-13) 

then  the  expected  value  of  J is  minimized  and  contains  a 

term  that  is  quadratic  in  the  initial  state  mean  given  by 

x*K(t  )x  . 
o 00 

It  should  also  be  noted  that  substitution  of  (2-12) 
into  (2-11)  and  (2-9)  identifies 

F(t)  - A ( t ) - B(t)R"1(t)BT(t)K(t)  (2-14) 


(2-14) 


N(t)  ■ Q(t)  + K(t)B(t)R-1(t)BT(t)K(t)  . 


(2-15) 


It  was  also  observed  in  [2]  that  under  the  conditions 


of  (2-12)  and  (2-13) 


K(t0)  « H(tQ,l) 


(2-16) 


Indeed  if  U(t)  had  been  selected  as 


u(t)  - -R_1(t)BT(t)H(t,l)x(t) 


(2-17) 


then  substitution  of  (2-17)  into  (2-10)  and  (2-12)  would 


lead  to 


F ( t ) - A ( t)  - B ( t ) R-1 ( t ) BT ( t ) H ( t , 1 ) 


(2-18) 


N(t)  ■ Q(t)  + H(t,l)B(t)R'‘1(t)BT(t)H(t,l)  • 


(2-19) 


Substitution  of  (2-18)  and  (2-19)  into  (2-6)  yields 


H(t,l)  - - H ( t , 1 ) A ( t ) - Aa (t) H (t , 1) 

+ H(t,l)B(t)R-1(t)BT(t,l)  - Q (t) 


(2-20) 


If  one  were  to  form  a linear  combination  of  cumulants 
of  J then  such  a combination  would  contain  a term  of  the 
form 


,c^H(t  0,i)]  (2-21) 


x^Cc^HU^l)  + a2H(tQ,2)  + 
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In  [l]  Hartwig  argues  that  if  one  were  interested  in 
selecting  u(t)  such  that  several  statistics  of  J would  be 
affected  in  a prescribed  way  then  it  might  be  reasonable  to 
select  u(t)  as 


U(t)  - - R_1(t) BT(t) [a1H(t,l) 

+ a2H(t,2)  + a4H(t,Z)  ]x(t)  . 


(2-22) 


He  subsequently  demonstrated  by  several  examples  that 
this  conjecture  was  reasonable  and  good  performance  could  be 
achieved  in  this  sense.  Substitution  of  (2-22)  into  (2-6) 
and  (2-7)  yields  the  class  of  Riccati  equations  of  interest 
in  this  work. 

For  this  initial  study  of  the  properties  and  character 
istics  of  these  equations  only  the  first  two  equations  are 
studied.  Without  loss  of  generality  is  selected  to  be 
1 and  a2  is  replaced  by  a. 

H (t , 1)  - -AT(t)H(t,l)  - H ( t , 1 ) A ( t ) - Q ( t ) 

- a2H(t,2)B(t)R-1(t)BT(t)H(t,2)  (2-23) 

+ H ( t / 1)  B ( t ) R”1  ( t)  BT  ( t ) H ( t , 1)  , 

and 

H ( t , 2 ) - -AT(t)H(t,2)-H(t,2)A(t)+H(t,l)B(t)R"l(t)BT(t)H(t,2) 
♦ H(t,2)B(t)R_1(t)BT(t)H(t,l) 

+ 2<xH(t,2)B(t)R’1(t)BT(t)H(t,2)-H(t,l)  5(t)H(t,l)  , 
with  the  boundary  conditions  (2-8)  and  (2-9) . (2-24) 


CHAPTER  III 


MATRIX  RESULTS 


For  a first  look  at  the  properties  of  (2-23)  and  (2-24) 

let 

[t0,  tf]  - [0,  10]  (3-1) 

and  select  the  coefficients 

‘-1  1 0 
A (t ) -000 

0 -1  -1 


(-3-2) 


Q(t)  » 0 (the  zero  matrix),  (3-3) 

B(t)  ■ R ( t)  * 3(t)  - I (the  identity  matrix),  (3-4) 
and  final  values 

H (10 , 1)  - 1,  (3-5) 

H (10 , 2 ) - 0.  (3-6) 

Note  that  the  Pair  [a,b]  is  controllable.  Figures  (3.1)  to 
(3.4)  contain  the  trajectories  of  the  eigenvalues  of  H(t,l) 
and  H (t , 2)  for  a - 0,  1,  2 and  4 respectively.  These 


eigenvalues  of  H(t,l)  are  designated  by  A^(t),  i * 1,  2,  3. 
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Those  of  H (t , 2)  are  correspondingly  designated  by  X^2(t), 
i * 1,  2,  3.  Note  that  for  small  values  of  a,  the  trajec- 
tories are  well-behaved.  Indeed,  selection  of  an  arbitrarily 
large  time  interval  will  result  in  steady  state  solutions  in 
negative  time.  However,  for  larger  values  of  a,  it  appears 
that  steady  state  solution  may  not  exist  and  the  trajectories 
may  be  unbounded  on  the  half  line. 

Another  interesting  property  is  that  the  trajectories 
are  non-negative  for  all  t.  It  is  interesting  to  note  that 
this  is  a general  property  of  (2-23)  and  (2-24)  which  can 
easily  be  shown. 

III-B  Non-negative  Definiteness 

It  is  claimed  that  the  matrix  solutions  to  (2-23)  and 
(2-24)  with  the  final  values  (2-8)  and  (2-9)  are  positive 
semi -definite  (non-negative  definite)  for  all  t.  To  see 
this  select  any  ecRn,  e / d (the  null  vector).  Place  H(t,2) 
in  a quadratic  form  in  e as 

eTH2e  - - eTA(t)H2e  - eTH2AT(t)e 

♦ eTH1B(t)R-1(t)BT(t)H2e 

+ eTH2B(t)R_1(t)BT(t)H1e  (3_7) 

+ 2aeTH2B(t)R~1(t)BT(t)H2e  - eTH1EH1  e 
where  H2  means  H(t,2)  and  means  H(t,l). 

* 
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It  is  easy  to  see  that  if  R(t)  , Q(t),  H ( t ) and  the 
final  value  are  symmetric  then  the  matrices  H(t,l)  and  H(t,2) 
are  symmetric  for  all  t.  If  the  symmetric  H is  ever  non- 
negative definite  then  a non  matrix  H2  can  always  be  found 
such  that 

H2  * *2  **2  (3-8) 

Then 

eTH2  e » eTH2H2  e » | | H2e ( | 2 (3-9) 

which  is  equal  to  zero  if  and  only  if  H2e  is  the  zero  vector. 
Now  consider  the  H2  trajectory  backward  in  time  from  t * t^. 
Initially  (finally)  H(t^,2)  is  the  zero  matrix  so  from  (3-7) 

9 

e?H(tf,2)e  * -eTSE(tf)Se  < 0 (3-10) 

since  E(t^)  is  non-negative  definite.  This  means  that  as  the 

trajectory  of  H2  is  followed  backward  in  time  from  t = t^, 

H2  initially  becomes  no  less  negative.  Under  reasonable 

smoothness  assumptions  on  the  coefficients  of  (2-24) , the 

solution,  H2 , is  continuous  implying  that  the  scalar  quantity 
T 

e H(t,2)e  is  also  continuous.  Now  consider  two  points  in 

time  t^  < t2«  It  follows  that  in  order  for  eTH(t,2)e  to  be 

T 

negative,  given  that  e H(t2,2)e  is  positive,  there  must  exist 
a t2  with  t^  < t^  < t2  such  that 

eTH(t3,2)e 


■ 0 , 


(3-11) 
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But  from  previous  arguments  applied  at  t = whenever  (3-11) 

is  satisfied 

eTH(t3,2)e  <.  0.  (3-12) 

T 

Thus  e H2e  equal  to  zero  is  a reflective  barrier  which 
cannot  be  penetrated.  Thus  H2  is  always  non-negative  definite. 
Similar  arguments  will  also  show  that  is  non-negative 


definite . 


CHAPTER  IV 


SCALAR  RESULTS 

From  this  point  on  the  sense  of  time  in  equations 
(2-23)  and  (2-24)  will  be  reversed.  This  is  equivalent  to 
negating  the  right  sides  of  (2-23)  and  (2-24) . With  this 
change,  the  equations  become  forward  time  equations  and 
the  content  of  this  chapter  is  devoted  to  examination  of 
steady  state  solution  questions  for  scalar  and  with 
constant  coefficients.  Rewriting  (2-23)  and  (2-24)  as  scalar 
forward  time  equations  with  constant  coefficients,  one  ob- 
tains 

Hx  - 2aHx  +c?dHj  - dH2  + q , (4-1) 

and 

H2  - 2aH2  - 2dH1H2  - 2ctdH2  + 5H2  , (4-2) 

where  for  sir..  ^ity 

d * b2r‘1  . (4-3) 

Define  new  dependent  variables,  V and  W by 

W ^ , (4-4) 

and 

V i H2e~2at.  (4-5) 

Substituting  for  and  H2  in  (4-1)  and  (4-2)  yields 

W ■ (-dW2  +a2dV2  + qe  ^at)e2at  , (4-6) 
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and 


V » ( - 2 dVW  - 2daV2  + = W2)e2at. 


Define  the  new  independent  variable  x by 
t 


x(t)£  | e2axdr  * [ e2at  -l] . 

o 


Then 


e2at  » 2ax (t ) + 1, 


next  let 


and , 


W*  ■ dW/dx  * (dt/dx)  * We  2at, 
o t 


V - dV/dx  = ~ (dt/dx)  » Ve"2at. 

0 * 


Now,  rewrite  (4-10)  and  (4-11)  in  the  form 
W'  » -dW2  + a2dV2  + q(2ax  + 1)  " 2 
V'  - -2dVW  - 2adV2  + SW2  , 
subject  to  the  initial  conditions 

-2at| 


W(0)  - W(x)i  - H.  ( t)  e 


X-0  ft 


Hx  (0 ) 


S, 


and 


7(0)  - V (x)|  - H2(t)e 


-2ati 


i 

x-( 


= H2(0)  * 0 


fc-0 


Define  a new  dependent  variable  U by 

U A V/W 


(4-7) 

(4-8) 

(4-9) 

(4-10) 

(4-11) 

(4-12) 

(4-13) 

(4-14) 

(4-15) 


(4-16) 
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Then 

0(0)  =»  V(0)/W(0)  = 0,  (4-17) 

and 

a = UW.  (4-18) 

It  follows  that 

U*  - (-1/W) [2adU2W  - EW2  + dW2U 

+ a2dU3W2  + qU(2ax  + 1)  “2]  . (4-19) 

Equations  (4-12)  and  (4-19)  with  boundary  conditions  (4-14) 
and  (4-17)  provide  a mechanism  for  examining  steady  state 
properties  of  (4-1)  and  (4-2).  Letting  S =*  1,  Figures  (4.1) 
through  (4.5)  contain  U-W  phase  plane  plots  for  several  dif- 
ferent values  of  a,  d,  q and  E.  Interpretation  of  these 
plots  is  enhanced  by  observing  that  (E/4d,  W)  with  W arbi- 
trary is  an  equilibrium  point  of  (4-12)  and  (4-19)  in  the 
V-W  phase  plane  if  a a 4d/E . 

On  all  of  these  plots  for  0 < a < 4d/E,  the  phase 
plane  trajectories  approach  W =»  0 implying  the  existence  of 
a steady  state  solution  for  H^.  The  value  of  U approached 
yields  the  corresponding  steady  state  solution  for 
since 

0 - H 2/Hx  . (4-20) 

For  a = 4d/=  no  steady  state  is  reached  for  but 
the  time  trajectories  of  and  H2  are  in  proportion  as 
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t - ® by  relation  H(t,2)  - (H/4d) H (t , 1) . That  is,  U ap- 
proaches E/4d.  For  a > 4d/E , numerical  instability 
occurs . 

Thus  it  may  be  concluded  that  steady  state  solutions 
to  the  Riccati  equations  studied  herein  may  exist  but  only 
for  limited  ranges  of  the  parameter  a. 


CHAPTER  V 


SYSTEM  EXAMPLES 

Now  that  some  of  the  properties  of  the  Riccati  equa- 
tions have  been  observed,  it  would  be  interesting  to  see 
the  properties  of  systems  that  contain  these  Riccati  equa- 
tion solutions  as  parameters.  In  this  chapter  one  example 
of  linear  estimators  and  two  examples  of  linear  feedback 
controllers  that  contain  Riccati  equation  parameters  are 
presented. 

In  both  example  classes,  the  gains  that  are  normally 
a function  of  the  classical  Riccati  equation  are  replaced 
by  gains  with  the  term  replaced  by  the  weighted  sum  of  the 
two  Riccati  solutions  studied  herein.  The  weighting  param- 
eter is,  of  course,  the  parameter  a,  that  appears  in  the 
actual  equations.  Note  that  when  a = 0 these  new  classes  of 
controllers  and  estimators  reduce  to  the  classical  cases. 
The  example  studies  here  are  carried  out  primarily  as  a 
function  of  the  parameter  a. 

Example  _1:  Filtered  estimate  of  nonlinear  process 

Consider  the  nonlinear  system 

x(t)  = sin(x(t))  , (5-1) 

with  measurement 

y (t)  = x ( t ) + 0(t)  , (5-2) 
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where  9 (t)  is  zero-mean  and  white  with  unity  covariance.  The 
initial  condition  for  (5-1)  is  random  with  assumed  mean  of 
zero  and  unity  covariance. 

In  this  example  it  is  desired  to  estimate  x(t)  based 
upon  observations  y(r)  , 0 <.  t <.  t.  To  accomplish  this  the 
system  (5-1)  is  first  linearized  to  give  a linear  model 

x(t)  =•  x(t)  , (5-3) 

y (t)  =*  x ( t ) + 9 (t)  . (5-4) 

Based  upon  this  model,  which  is  a good  approximation 
only  for  small  x(t),  a linear  filter  is  designed  utilizing 
the  classical  structure  shown  in  Figure  (5.1)  with  the 
filter  gain,  ? (t) , selected  as 

P(t)  = H1(t)  + aH2(t)  (5-5) 

where  ^ (t)  and  H2  (t)  are  solutions  to 

2 2 , 

- 2H1  + a H2  - , (5-6) 

and 

H2  - 2H2  - 2H1H2  - 2aH2  + , (5-7) 

with 

Hj^CO)  - 1 , 

and 

H2  (0)  * 0 . 


(5-9) 
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Note  that  (.5-6)  and  (5-7)  are  the  duals  in  the  traditional 
control/filtering  sense  of  (2-23)  and  (2-24) . There  is  one 
subtlety,  however,  in  that  there  is  no  dual  for  the  E(t) 
coefficient  in  (2-24).  Consequently  it  becomes  a design 
parameter  and  has  arbitrarily  been  set  to  unity  in  this 
example . It  should  be  noted  that  if  this  coefficient  is 
set  to  zero,  then  Hj  will  be  identically  zero  and  solution 
of  (2-23)  and  (2-24)  for  any  value  of  a is  equivalent  to 
solution  with  a » 0 . It  is  interesting  to  note  that  the 
steady  state  value  for  x(t]  in  (5-1)  is  it  and  so  (5-3)  is 
not  a good  approximation.  Despite  this  the  linear  filter 
performs  quite  well.  In  Figure  (5,2)  the  actual  state 
trajectory  evolving  from  a randomly  selected  initial  condi- 
tion of  xq  » 0.8  and  filtered  estimate  trajectories  for 
a«  0,  1,  and  2 are  shown.  Note  that  for  a = 0 which  is  the 
classical  Kalman  filter, the  estimate  is  not  as  good  as  for 
th.e  cases  a » 1 and  2.  In  these  cases,  however,  there  tends 
to  be  more  oscillatory  behavior. 

Example  2 : A Second  Order  Linear  Regulator 

Control  System 

In  this  example  consider  the  linear  system 

x(t)  - Ax (t ) + Bu(t)  (5-10) 


with 
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0 1 
0 0 


(.5-11) 


B 


0 

1 


(5-12) 


and 


x (0) 


1 

1 


(5-13) 


The  control  action  u is  selected  to  be 

u(t) 


- r”1BT[H1  + aH2]x(t) 


(5-14) 


where  and  H2  are  the  steady  state  solutions  to  equations 
(2-23)  and  (2-24)  with 


0.25 

0 


0 

0.25 


1 0 
0 1 


(5-16) 


and 


(5-17) 


In  Figures  (5.3)  and  (5.4)  the  state  trajectories 
of  the  closed  loop  system  are  shown  fora*  0,  5 and  9..  This 
example  first  appeared  in  [2]  where  the  system  was  noisy 
and  5 was  the  noise  covariance.  In  [2]  control  laws  similar 
to  those  here  were  selected,  but  only  over  a finite  time 


19 


interval.  It  was  demonstrated  in  [2]  that  these  controllers 
were  good  in  a second  order  statistical  sense.  The  question 
addressed  here  is  that  of  the  asymptotic  stability  of  (5-10) 
under  the  feedback  (5-14).  After  all,  in  order  for  the  con- 
trol laws  suggested  by  Hartwig  in  [2]  to  be  acceptable,  they 
must  not  only  have  good  statistical  properties , but  should 
also  stabilize  the  system. 

Note  again  as  in  Example  1 that  when  a 0 the  feed- 
back gain  collapses  on  a classical  structure.  In  this  case 
it  is  the  classical  linear  regulator  which  is  known  to  be 
asymptotically  stable.  As  can  be  seen  from  the  state  trajec- 
tories in  Figures  (5.3)  and  (5.4).  The  closed  loop  system 
is  also  stable  for  the  non  zero  values  of  a.  Indeed,  it 
appears  from  these  time  domain  pictures  that  for  larger 
the  system  is  more  stable  than  for  the  case  a = 0. 

To  obtain  a complete  picture  of  the  stability  of  the 
feedback  system  look  at  Figure  (5.5)  where  the  loci  of  the 
closed  loop  poles  of  the  system  are  plotted  in  the  complex 
plane  as  a function  of  o. 

Table  I contains  several  classical  second  order  system 
parameters  £ , wn,  T^  and  % overshoot  as  a function  of  a.  It 
is  most  interesting  to  note  that  for  a > 9 numerical  insta- 
bility arose  in  attempting  to  find  steady  state  solutions 
to  the  Riccati  equations  so  instability  of  the  closed  loop 
control  system  was  never  attained. 
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Example  3:  A Third  Order  Linear  Regulator 

Control  System 

A last  example  of  third  order  will  again  demonstrate 
similar  stability  properties  as  a function  of  a.  The  system 
is  the  two-vehicle  problem  described  in  [3].  That  is  a 
linear  system  with 


A ■ 


-1 

1 

0 


0 0 ' 
0 -1 
0 -1 


(5-18) 


and 


B - 


‘ 1 0 * 
0 0 
0 1 . 


(5-19) 


The  feedback  control  law  is  again  chosen  as  in  (5-4)  with 


Q - 


0 0 ' 
10  0 
0 0 


(5-20) 


R - 


‘ 1 O' 

. 0 1 . 


and  E chosen  first  as 

’ 1 0 O' 

E - 0 10 

.0  0 1 . 


(5-21) 


(5-22) 
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State  trajectories  for  this  case  with 


(5-23) 


appear  in  Figures  (5.6),  (5.7)  and  (5.8)  for  a ■,  0.7  and 
1.2.  Changing  3 to 

5.0  0 O' 

0 5.0  0 (5-24) 

0 0 5.0. 


yields  the  trajectories  in  Figures  (5.9)  , (5.10)  and  (5.11). 

Corresponding  pole  loci  for  the  closed  loop  systems  are  shown 
in  Figures  (5.12)  and  (5.13)  with  accompanying  parameters  in 
Tables  II  and  III. 


CONCLUSIONS 


Future  research  on  the  new  class  of  Riccati  equations 
should  address  the  question  of  global  existence  of  solutions. 
Numerical  behavior  of  the  equations  indicates  that  global  exis- 
tence may  be  subject  to  the  value  of  the  a parameter.  The 
specific  relationship  between  open-loop  system  properties,  the 
Riccati  equations  and  closed  loop  system  performance  needs  to 
be  analytically  explored.  The  relationship  between  the  value 
of  a and  the  relative  stability  of  a closed  loop  control  system 
designed  via  these  equations  is  only  one  example  of  this. 

A detailed  study  of  estimators  ffilters  in  particular) 
designed  via  the  new  equations  needs  to  be  carried  out.  Whe- 
ther' or  not  the  properties  observed  here,  in  nonlinear  applica- 
tion are  general,  is  a completely  open  question. 

A full  understanding  of  the  duality  between  the  fil- 
tering and  control  contexts  needs  to  be  developed. 
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APPENDIX  A 

£,  Wn,  Td,  and  % Overshoot 
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TABLE  I 

£,  Wn,  Td  (The  Period  of  Oscillation)  and  % Overshoot 
as  the  Function  of  a for  Example  3 


a 

*i 

' X2 

e 

can 

Td 

T 

overshoot 

O.'O 

-0.8660 

10.500Q j 

0.866025 

1.0 

12.57 

0.43 

0.1 

0. 89410.0 

1.0 

14.03 

0. 19 

0.2 

-0.8210 

to. 3894j 

0.9034 00 

0.9087 

16.13 

0.13 

mm 

0.94700Q 

0.9999 

19.56 

0.095 

0.4 

0.972350 

0.9999 

26.90 

0.0002 

mm 

-0.9971 

-0..0747J 

0.997198 

0.9999 

83.99 

3S 

0.6 

-0.8122 

/-I .0216 

1.006580 

0.7 

-0.7393. 

,-1.0459 

1.015000 

1.0 

-0.6180 

, — 1 .1179 

1.044220 

mm 

-0.4353 

,-1.3662 

1.168000 

mm 

-0.3364 

,-1.6540 

1 .334000 

4.0 

-0.2652 

,-2.0175 

1.560000 

mm 

-0.2077 

,-2.5110 

1 .880000 

mm 

-0.1589 

,-3.2418 

2.368500 

7.0 

-0.1134 

,-4.4629 

3.2154Q0 

8.0 

-0.0721 

,-6.9631 

4.961820 

mm 

-0.0318 

,-15.085 

10.91312 

TABLE  II 

£,  Wn,  Td  and  % Overshoot  as  the  Function  of  a for 
Example  3 with  the  State  Noise  Covariance 
Equal  to  Unity 


un  Td 


2.6108 

4.460 

0.7400 

2.8520 

4.230 

0.3800 

3.0344 

4.150 

0.4300 

3.1770 

4.210 

0.2700 

3.2752 

4.490 

0.1300 

3.2946 

5,330 

0.0275 

3.0654 

62.255 

i 

1 .77xl0~39 
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TABLE  III 

Wn,  Td  and  % Overshoot  as  the  Function  of  a for 
Example  3 with  the  State  Noise  Covarinace  Equal  to  51. 


a 

X1  ' X2 

C 

con 

Td 

% 

overshoot 

0.0 

-2.1980  -1.40903 

0.841870 

2.610800 

4.460 

0.74000 

0.1 

-3.0760  -1.17903 

0.833759 

3.294200 

5.330 

0.02750 

o.n 

-3.0930  -0.9651 j 

0.954600 

3.240000 

6.509 

0.00424 

0.12 

-3.0145  -0.5560j 

0.983410 

3.065300 

11.300 

4.01xl0'6 

0.122 

-2.9586  -0.4000J 

0.990784 

2.985517 

15.700 

8. 098x1 0'9 

0.123 

-2.9106  -0.2878J 

0.895146 

2.924790 

21.830 

1 .sgxiO’12 

0.124 

-2.7920  -0. 05233 

0.999820 

2.782490 

1.2xl02 

1.94xl0'71 

~r~~r 


APPENDIX  B 

Time  Domain  Trajectories,  Phase  Plane  and 
Closed  Loop  Pole  Locations 


28 


Eigenvalues  Eigenvalues 


Fig.  (1.1)  The  eigenvalue 
trajectories  for  equations 
(2-23)  and  (2-24)  for  the 
conditions  (3-1)  to  (3-5) 
a = 0. 


Ao'^X 


Fig.  (1.2>  The  eigenvalue 
trajectories  for  equations 
(2-23)  and  (2-24)  for  the 
conditions  (3-1)  to  (3-5) 
a = 1 . 


> 1-5' 


/wKJ 

\ 


Fig.  (3.3)  The  eigenvalue 
trajectories  for  equations 
(2-23)  and  (2-24)  for  the 
conditions  (3-1)  to  (3-5) 


Fig.  (3.4)  The  eigenvalue 
trajectories  for  equations 
(2-23)  and  (2-24)  for  the 
conditions  (3-1)  to  (3-5) 


a =4.0 


(0.25,0.81) 


Fig.  (4.1)  The  U-W  phase 
plane  plots  for  equations 
(4-12)  and  (4-19)  with  a=l, 
d=l,  q=0 , 5=1. 


(0.125,0.81) 


»a=0 .0 
o=2.0 
X o=4.0 
\ X a=6.0 


Fig.  (4.2)  The  U-W  phase 
plane  plots  for  equations 
(4-12)  and  (4-19)  with  a=l 
d=lj  q=0,  E =0.5. 


(0.25,1.19) 
i^o  =4.0 

(\^a=i.o 


(0.25,1.69) 


-a=4 . 0 


Fig.  (4.3)  The  U-W  phase 
plane  plots  for  equations 
(4-12)  and  (4-19)  with  a=l, 
1*1,  q*l,  2*1. 


Fig.  (4.4)  The  U-W  phase 
plane  plots  for  equations 
(4-12)  and  (4-19)  with 
a=l,  d=0 . 5 , q=0 . 6 , z =0.5. 


0.8 


Fig.  (5.11)  Time  domain  Fig.  (5.12)  Closed  loop 

trajectories  of  X3  in  pole  locations  as  a 

Example  3 with  H = 51.  function  of  a for  Example  3 

with  5*1. 


Fig.  (5.13)  Closed  loop  pole 
locations  as  a function  of  a 
for  Example  3 with  5 - 51. 


